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Description 

[0001] This invention relates to an apparatus and a method for high fidelity reconstruction of an observed sample 
and for visual reconstruction of the data. 

5 

DEFINITIONS AND ABBREVIATIONS 
[0002] 

w EM - electron microscopy 

ET - electron tomography 
CL - common lines method 

F(X) - density values at points (x.y.z) in a 3D reconstruction 
15 - m(X) - a prior prejudice distribution 

C - A chi-squared statistic - a sum of squared differences between a projected 3D reconstruction and an observed 
projection divided by a measured variance of the observations. 

W(X) - an interpolation used for convolution operation given by the fact that a 3D density is built on a 3D grid, said 
grid values defining a continuous function through a trilinear interpolation and decreasing the resolution somewhat 
20 because of decreased bandwidth. 

p(') - a projection operation to the ith view 

a pupil function based on the deflection of electrons by the iris diaphragm. 
CTF - a contrast transfer function 

PSF - a point spread function, by which each projected image must be subjected to convolution. PSF in Fourier 
25 space is the pupil function * CTF. 

T('>(X,x(')) - a smearing function, based on the projected PSF 
0<')(X , Y ,x (•)) - an overlap function 
* - multiplication 
® - convolution 

30 

INTRODUCTION 

[0003] In all data acquisition and signal processing systems resolution is a key factor. The degree of resolution is a 
direct measure of the size of the smallest detail that can be reproduced. The higher the resolution the better the re- 
35 cording. Resolution, however, is not all. A data acquisition system reacts, not only to the wanted quantity, but also to 
random processes such as noise and other interferences. When these interferences are of significant magnitude as 
compared with the resolution of the system, care must be taken in order to ensure that only meaningful information is 
extracted from the recorded data. 

40 BACKGROUND OF THE INVENTION 

[0004] Increased resolution and filtering are common practice in data acquisition and processing. The basics of this 
can be found in any university textbook on the subject, and numerous inventions have been made in this field. 
[0005] The present invention relates to data acquisition and processing systems of such dimensions that the sheer 

45 amount of data poses serious technical problems in implementing conventional techniques. A particular field where 
these problems occur is the field of three-dimensional (3D) imaging of small objects at high resolution. In this case, 
the mere size of the calculations needed presents substantial technical problems. This calls for the use of more refined 
methods. Once established, such methods could have a broad range of applications. Thus, the teaching from them 
could also be transferred to other dimensions, for instance 2D and 1D applications. Also, 3D imaging of large sized 

50 objects at high resolution is an interesting application. 

[0006] The fundamentals of for instance 3D reconstruction were investigated in the beginning of this century math- 
ematically by Johan Radon. The idea of 3D imaging of small objects (molecules) was shown and conceived by Aaron 
Klug and coworkers, but was not implemented for single particles. 

[0007] Several techniques have since been developed for reconstructing 3D images of different types of objects. 3D 
55 imaging of macromolecular complexes lacking symmetry still has technical problems to be solved. In this case, com- 
puter power is not enough. The equations to be solved are of such proportions that earlier attempts to solve the technical 
problem of computing the 3D image reconstruction, from a series of recorded 2D projections from an electron micro- 
scope, so that it can be visually displayed as a 3D image, have had limited success due to the limited accuracy and 
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high noise levels. 

[0008] The technical problems concerned with the reconstruction of 3D images of small objects at high resolution 
from a number of 2D projections and with a high signal to noise ratio, have not yet been solved in prior art. 
[0009] One technical problem to overcome is that of the contrast transfer function (CTF) of the electron microscope, 
5 or any other input means that is used. CTF is dependent on microscope and focus and most of the parameters are 
linked to the machine to be used. This makes it difficult to achieve a quantitatively correct reconstruction of the measured 
quantity. 

[0010] The noise levels in 3D imaging have been too high for single particle molecular imaging. Using a larger set 
of tilts increases the risk of radiation damage to the specimen. Averaging has limited effect on artefacts in recorded 

10 data. These effects are reduced by averaging but are still present. 

[0011] Gradient methods have been implemented and used more recently in electron microscopy (EM) 3D recon- 
structions. However, in EM applications, as well as in some medical radiation therapeutical applications, the use of 
gradient refinement methods for image restoration seldom results in a substantial improvement in the quality or reso- 
lution of the refined 3D reconstruction due to the fact that a large portion of data is missing, as the reconstruction is 

15 made from a limited tilt series. 

[0012] Another technique of 3D reconstruction is to project onto convex sets, which utilises an envelope that engulfs 
the 3D reconstructed object. Density modulations outside the envelope are regarded as artefactual and are reset to a 
constant value during the iteration cycles. The iterative refinement cycles proceed until the density modulations become 
small. The degree of improvement in a 3D reconstruction provided with this method has not been unambiguously 

20 established by comparison with an objective model. 

[0013] In the field of a 3D image reconstruction the resolution which it is desirable to obtain could require such a 
large number of recorded data sets that calculations could be difficult to accomplish and this thus sets a practical limit 
for the resolution. The use of symmetry of a crystalline specimen or internal symmetry of a specimen reduces the size 
of the calculations needed to a technically manageable level. However, not all of the objects in question have internal 

25 symmetry, crystalline symmetry, or symmetric arrangement, and it would therefore be advantageous to have a method 
of data acquisition and processing, including 3D reconstruction, which is not dependent on symmetry. Prior art solutions 
have not yet overcome the noise problem from intermediate (below 15A) to high (below 7A) resolutions without the 
need for symmetry to manage the calculations. 

30 DESCRIPTION OF RELATED ART 

[0014] A method of multi-scale reconstruction of the image of the structure of a body at an increased speed using 
iterative algorithms is disclosed in the U.S. Patent No. 5,241,471. The algoriths are constructed such that the image 
of the object to be reconstructed is done on a structure, for instance a grid, having a finer and finer finesse from iteration 
35 to iteration. In this way the calculations could be done very fast. Thus, it will not work with fine details from the start 
and do not count on components having a resolution higher that the resolution of the structure (grid) actually in the 
iteration. No density distribution for individual grid points is done. The description of the sample on the grid is thus not 
changed from iteration to iteration, only its resolution. 

[0015] An approach for providing a solution of this problem is to use maximum entropy. This is described by G.J. 

40 Ericksom & C.R. Smith, Maximum-Entropy and Bayesian methods in science and engineering, Volume 1:Foundations 
1-314 (Kluwer Academic Publishers, Dordrecht, The Netherlands (1988)), by C.R. Smith & W.T. Grandy Jr, Maximum- 
Entropy and Bayesian methods in inverse problems, p 1-492 (D. Reidel Publishing Company, Dordrecht, Netherlands 
(1985)), and by B. Buck & A. Macaulay, 220 (Oxford University Press, New York (1991)). A maximum entropy 3D 
reconstruction has the property of being maximally noncommittal with regard to missing information (E.T. Jaynes, 

45 Physical Review 106, p 620-630 (1957)), and thus the maximum entropy method could serve as a powerful method 
to remove some of the detrimental effects caused by missing projections in ET reconstructions. 
[0016] A method to perform a 3D reconstruction of an object with high resolution is described by Ali Mohammad- 
Djafari et al, "Maximum Entropy Image Reconstructions in X-Ray and Diffraction Tomography", IEEE Transactions on 
Medical Imaging Vol. 7 (Dec. 1988) No 4, New York USA, PG. 345-354. 

50 [0017] The argument and the applications in this referens refer only to a reconstruction from ID to 2D, because of 
the idea that in medical imaging one reconstructs each circular "slice" independently from the others. Thus only 2D 
slice reconstructions are performed and the 3D is provided by adding the slices together. Thus the slices are treated 
as independent from each other. This method of treating the slices as independent gives unpredictable result when 
deconvoluting the real 3D point-spread function and possible line broadening. The result provided with the method 

55 described in this article can therefore not be quantitatively correct. When using the Poisson statistical properties of the 
image this is then not correct since it emanates from the a large part of the specimen, since the treatment of it is as if 
it were a local phenomenon restricted to a 1 D line. 

[0018] The implementations of the maximum entropy principles in ET 3D reconstructions have aimed at maximising 
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the entropy while under the constraint that the reduced chi-squared statistic be equal to 1 .0 as suggested by M. Bartn. 
R.K. Bryan, R. Hegerl & W. Baumeister, Scanning Microsc. Suppl. 2, p 277-284 (1988) and by M.C. Lawrence, M.A. 
Jaffer & B.T. Sewell, Ultra microscopy 31, p 285-301(1989). 

[0019] The usefulness of assigning low weight to improbable situations,-so that the iterative solution of the problem 

5 becomes that of maximising the entropy relative to the non-informative prejudice, i.e. an estimated 3D prejudice dis- 
tribution of the density, under the constraints of making the reduced chi-squared statistic equal to +1, has been shown 
by Gerard Bricogne in the article 'Maximum entropy and the foundations of direct methods' in Acta Crystallographica 
A40, pp410-445 (1984), which discloses an algorithm used for constrained entropy maximisation for crystallographic 
phase refinement against reciprocal space data. However, the mathematics shown by G. Bricogne can not be applied 

10 on the tomographic problem. His method is a development of so called direct methods within crystallography which 
give a generalized solution of a statistical problem and does not deal with imaging problems. A diffraction pattern is 
provided which is the diffraction data from crystallographic examinations giving focal planes, and thus no image is 
provided. When several generated focal planes are collected, for instance on a photographic film or detector, then the 
phase information of the different arriving waves is lacking.The crystallographic problem is aimed at finding the phase 

15 information so that an image can then be calculated. 

[0020] Thus the problem in crystallography is different to that of 3D imaging of macromolecular complexes lacking 
symmetry, which is solved by the invention. However, the tools that have been developed to be used for providing a 
high fidelity reconstruction in real space could be the knowledge that it is important to have a constrained maximum 
entropy formalism and to use Taylor expansion of the chi-squared statistic and the entropy as quadratic models. 

20 [0021] This kind of direct method described by Gerard Bricogne is directed to solve the phase problem in crystallog- 
raphy and is therefore not applicable in 3D imaging of macromolecules lacking symmetry. Therefore, the whole concept 
had to be totally redesigned in order to perform for the real space imaging problem. The algorithm used for constrained 
entropy maximisation was thus first devised for crystallographic phase refinement against reciprocal space data. It was 
adapted to the tomographic situation by replacing the Fourier transformation which relates molecular model to diffraction 

25 data in the crystallographic setting by line projections in real space. However, it could not handle projections at any 
angle. The problem was to provide a method which was independent of specific data sampling strategies like single- 
axis tilt series, conical tilt series, random tilt series, or tilts from symmetric samples where the projection angles could 
be calculated later (as is the case for the adenovirus). This technical problem has now been solved according to the 
invention. 

30 

OBJECTS OF THE INVENTION 

[0022] Is is an object of the invention to provide high fidelity reconstruction of an observed sample, for instance by 
3D imaging, practically independent of the size of the object. 

35 [0023] It is an object of the invention to overcome the technical problems mentioned above, concerned with 3D 
imaging of single particle macromolecular complexes at high resolution, with a high signal to noise ratio. 
[0024] It is a further object of the invention to provide a method and an apparatus which is independent of the object's 
symmetry and which also offers the possibility of removing detrimental effects during the reconstruction of missing data. 
[0025] It is also an object of the present invention to provide a 3D image reconstruction of single particle macromo- 

40 lecular objects lacking symmetry, with a high enough resolution to make it possible to reconstruct a 3D image of mac- 
romolecular objects lacking symmetry, using a series of 2D projections recorded with an electron microscope or a 
similar means. 

[0026] It is a further object of the present invention, to solve the technical problem of the high noise levels present 
in 3D imaging, thus providing a low noise 3D imaging. 
45 [0027] Another object of the invention, is to overcome the technical problem of the contrast transfer function of the 
electron microscope, or any other input means that is used, which makes it difficult to achieve a quantitatively correct 
reconstruction of the measured quantity. 

[0028] A further object of the present invention, is to minimise the detrimental effects of non-recorded data on the 
reconstructed image. 

50 [0029] Still a further object of the present invention, is to overcome the technical problem of imaging objects lacking 
symmetry. 

[0030] Another object of the invention, is to overcome the technical problem of defining the physical parameters 
needed for accurate and optimal reconstruction. 

[0031] Still another object of the invention is to provide an apparatus and a method which record, compute and 
55 present a true, i.e. a quantitatively correct, reconstruction of an observed sample with very high resolution. 

[0032] Another object of the invention is to provide a reconstruction easy to use for the reconstruction of samples in 
different dimensions, for instance 3D, 2D and 1D appplications. 
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SUMMARY OF THE INVENTION 

[0033] A solution to the objects of the invention is provided by a method and an apparatus for generating a high 
fidelity reconstruction built on a grid having grid points of an observed sample as defined in appended independent 
5 claims 1 and 14. The method of the invention comprises, inter alia, method steps directed to: 

provide an initial estimated distribution of the sample built on the grid; 
provide a blurred prior prejudice distribution using estimated data; 
calculate in an iterative process for each iteration: 

10 

* a new estimated distribution of the sample using comparison between the estimated distribution in the next 
preceding iteration and observed data of the sample, 

* a new prior prejudice distribution on the new estimate less blurred than the prior prejudice distribution in the 
next preceding iteration; 

15 

continue the iterations until the difference between the new estimated distribution and the next preceding estimated 
distribution is less than a predetermined condition. 

[0034] The blurring is preferably provided by making a Fourier transform of the estimated structure and by multiplying 
20 the coefficients of this Fourier transform with the Fourier coefficients of a gaussian and spherical filter. The blurred 
structure is normalised before it is used as the new prior prejudice distribution in the next iteration cycle. Moreover the 
following method steps are performed. 

a) providing several recorded observed data of the sample, each from a different aspect of the sample; 
25 b) providing a variance for individual observation grid points in each recorded observed data; 

c) calculating the reconstruction of the observed sample structure in the iterative process taking said initial esti- 
mated distribution as a first iteration reconstruction approximation; 

d) for each iteration: 

30 calculating said new prior prejudice distribution using the next preceding reconstruction result, 

calculating a reduced chi-squared statistic (C) using the immediately preceding calculated reconstruction, the 
recorded observations and said variance, and the grid, while maintaining normalization of the calculated re- 
construction, calculating an entropy (S) using the immediately preceding calculated reconstruction and the 
new prior prejudice distribution; 

35 

e) maximising the entropy under the constraint of driving the reduced chi-squared statistic towards +1, and pro- 
viding a new calculated reconstruction to use in the next iteration cycle; and 

f) presenting the reconstruction. 

40 [0035] Particularly, an apparatus and a method have been developed by which it is possible to record, compute and 
present a quantitatively correct 3D image of a macromolecular complex at high resolution on a visual display. However, 
the method as such is independent of size of the object and thus a correct 3D image of large sized objects could be 
done as well, for instance from tomographic recordings. 

[0036] The invention is a development of, and thus based on, a maximum entropy method adapted to tomographic 
45 principles. According to the invention the Fourier transformation (which relates model to data in the crystallographic 
setting) is replaced by line projections in real space. 

[0037] Thus, the maximum entropy distribution has, among other things, the property that no possibility is ignored, 
and that all possible situations are given a positive weight. 

[0038] A more powerful formulation than were proposed in the prior art is then to specifically make use of prior 
so information in the algorithm and assign low weights to improbable situations, so that the iterative solution of the problem 
is that of maximising the entropy relative to the non-informative prior prejudice, i.e. an estimated 3D probability distri- 
bution of the density under the constraint that the reduced chi-squared statistic goes towards +1. 
[0039] The maximum entropy 3D reconstruction has the property of being maximally noncommittal with regard to 
missing information, and could thus serve as a powerful method to remove some of the detrimental effects caused by 
55 the missing projections in ET (electron tomography) reconstructions. The implementations of the maximum entropy 
principles in ET 3D reconstructions have aimed at maximising the entropy under the constraint that reduced chi-squared 
statistic is aimed towards but does not go below +1. The reduced chi-squared statistic is the sum of the squared 
differences between the projected 3D reconstruction and the observed projections, each divided by the measured 
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variances of the observations, and divided by the number of gridpoints of the projection in question. The variance of 
the observation data is the stated knowledge of them, and therefore the reduced chi-squared statistic is not allowed 
decrease below + 1 , because then that would mean that there are differences which are more relevant than the variance 
of the data. The maximum entropy distribution also has the property that no possibility is ignored and that all possible 
5 situations are given a positive weight. 

ADVANTAGES OF THE INVENTION 

[0040] The inventive method is preferably performed by having the same resolution and using the same grid for each 
10 projection. The reason for having low resolution of the density distribution on the grid points at the beginning and a 
high resolution later on is that it describes the probable density distribution of the objects in space. Thus, it does not 
describe that the object itself is blurred, but that the probable density distribution on the grid of-it is blurred. After the 
first iteration a better adaptation has been done and because the entropy, for instance, has been higher the influence 
of the background has been lower. Therefore, a more narrow density distribution could be assumpted in the next 
15 iteration cycle. Thus, no finer finess regarding the grid is done as in the U.S. Patent No. 5,241,471 mentioned above. 
In the invention instead, the blurring is changed from a starting density distribution to give a better and better density 
resolution at the grid points from iteration to iteration. 

[0041] With an apparatus and a method according to the invention, it is possible to solve the technical problem of 
high resolution 3D imaging of macromolecular objects lacking symmetry by using a reconstruction procedure that 
20 exploits the prior information, thereby providing a model-free refinement, and producing a 3D structure whose projection 
convoluted with the point spread function of the microscope will adapt to the observed structure to give a low noise 3D 
reconstruction that is quantitatively correct. 

[0042] Thus, the results provided by the invention gives more reliable results than refinements based on overpara- 
metrised models. For instance, in the case of using diffraction, if the physics for the model refinement in not correct, 
25 then it is very difficult to come to an adequate result. This difficulty is avoided according to the invention. 

[0043] The key to solving these technical problems lies within the practical solving of mathematical equations. All 
prior attempts to reconstruct and visually display a true 3D image of single macromolecular complexes lacking symmetry 
have failed, due to the fact that equations technically solvable in practice have not been available. 
[0044] The awesome size of the calculations needed is reduced considerably by the use of an approximation that 
30 the value of the smearing function is 3 0 only for a limited number of grid points, and by continuous improvement of 
the prejudice distribution, which reduces the amount of iteration cycles needed. Also a reduced calculation expression 
is given which speeds up the calculation such that it can be evaluated in about 1 minute, instead of about 300 years 
of evaluation time without the reduced expression. Also, a higher calculation precision is provided when using the 
reduced calculation expression. A further detailed description and discussion about this feature is given below in the 
35 description of the preferred embodiment. 

[0045] An additional advantage of the invention is that it ends up close to the high theoretical resolution level given 
by Shannon's law. This means that the method according to the invention could be applied to any conventional data 
acquisition and processing system, for instance an over-sampled such, where it would provide both a higher resolution, 
due to the noise eliminating qualities of the method, than is possible using prior art methods. 
40 [0046] What has been further perceived according to the invention is that a multiplying by the coefficients of the 
Fourier transform of the PSF function and the projected 3D density must be done instead of a division of the Fourier 
coefficients of the observed image with the Fourier coefficients of the PSF function because it is by using such a division 
many of the errors in the previous attempts to achieve a true reconstruction have been introduced. Thus the invention 
creates a 3D-structure whose projections after blurring with the PSF function will be the same as the observed projec- 
ts tions. Thus, when the iterations are ended then a 3D-structure is provided in which practically no errors have been 
introduced. 

BRIEF DESCRIPTION OF THE DRAWINGS 

so [0047] For a more complete understanding of the present invention and for further objects and advantages thereof, 
reference is now made to the following description taken in conjunction with the accompanying drawings, given as an 
example only, and in which: 



Fig. 1 shows a functional block diagram of a preferred embodiment of an apparatus according to the invention. 



55 



Fig. 2A to 2C show flowcharts of preferred programs for the computer shown in fig. 1. 



Figs 3A, 3B and 3C show 2D projections firstly recorded by an electron microscope of an adenovirus particle. 
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secondly recorded from a reconstruction done according to a conventional method, and thirdly recorded from a 
reconstruction done according to the invention. 



Fig 4 shows a functional block diagram of a second embodiment of the invention. 



5 



Fig 5 shows a functional block diagram of a third embodiment of the invention. 

DETAILED DESCRIPTION 

10 Description of a preferred embodiment 

[0048] Fig. 1 shows a preferred embodiment of an apparatus for data acquisition and processing according to the 
invention, used in order to achieve a 3D image reconstruction at high resolution, of an object with no internal symmetry. 
It is independent of specific data sampling strategies and compensates for the different contrast transfer effects on the 
15 different views of a tilt series of projections thus handling projections at any angle, focusing on different depths or the like. 
[0049] The apparatus comprises a specimen preparation means 1 . The specimen is prepared, according to conven- 
tional methods, for viewing in a microscope 1 0, preferably an electron microscope but it could also be a light microscope 
or confocal microscope. 

[0050] The microscope 1 0 provides several 2D projections in digitized form of the specimen, preferably by scanning 
20 it in a scanning means 11 connected to or included in the microscope 10. When the microscope is an electron micro- 
scope the recordings could be made under different tilt angles of the object with respect to the electron beam. 
[0051] The method according to the invention could also be used in light microscopes. Then it is also possible to 
record projections focused at different depths in the specimen. 

[0052] The method could also be used when recording a plurality of identical particles by considering each recorded 

25 particle as a different projection of one sample particle. 

[0053] The digitized 2D projection data, i.e. density values, are supplied to a computer, wherein said data is first 
filtered in a filtering operation 13 to remove spurious inadequate signal jitter. Each 2D projection is an image and is 
also processed regarding geometry description for making the projections in the right directions, controlling image 
parameters etc in order to bring all the projections in harmony with each other. To compare projections to the observed 

30 micrographs, the densities in each projected image must also be scaled to a proper level for the i:th tilt by j > 



[0054] Thereafter, the processed data emanating from the microscope is scaled in a scaling operation 14 before it 
is transferred to a calculation operation 15 as observed micrographs. 

[0055] In the computer 12, the maximum entropy refined 3D reconstruction is calculated iteratively from said data, 
40 maintaining normalization and driving the reduced chi-squared statistic to +1 under the constraint of prior prejudice, 
also called prior knowledge, as will be further explained below. The refined 3D reconstruction is fed to a presentation 
means 16, preferably a visual display, where it is presented as a 3D image. 

[0056] Several help functions must be introduced into the computer 12 in order to enable it to make the iterative 
calculation to provide the high fidelity reconstruction. 

45 [0057] A point spread function, PSF 18, is a characteristic function of the apparatus, such as a product of a pupil 
function and an analytical CTF (Contrast Transfer Function) of the devices 10 and 11. These parameters comprise 
defocus, acceleration voltage and spherical aberration etc. The CTF could be supplied as an experimental curve or 
as parameters refined from a parametrized model instead, and is one of the required inputs for the calculations. PSF 
can be computed by the computer 12 from measured apparatus parameters or an already computed PSF regarding 

so the equipment actually used can be fed directly to the computer. Thus the block 18 comprises either measured appa- 
ratus parameters to be processed by the computer 12 or directly generated or already stored PSF. 
[0058] When focus variations have to be considered for thicker specimens, the 3D density can be split up in layers 
of necessary thickness with each layer convoluted in the manner mentioned above with its proper PSF. Finally the 
layers are simply summed together. 

55 [0059] Thus, when there is a variation across an image, then PSF is varying even within the image. In the case that 
there is a variation in the depth direction then the PSF is varying along the depth. If both said variations exist the the 
PSF is varying in both directions. The variation of the PSF causes no problems. 

[0060] A 3D grid 1 22 is also fed into the computer. The reconstruction, the estimation and the prior prejudice functions 




(i) 
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are then built on this grid. 

[0061] Another input 19 to the computer 12 is the digitized data of an estimated reconstruction, e.g. a 3D reconstruc- 
tion made with conventional methods, which is used as the starting value for the iterations. This input is written into 
and stored in the computer 12 prior to the input from the scanning device 11. The estimated structure is normalized in 

5 a normalization means 20 either before it is inserted in the computer or by the computer program. 

[0062] The computer makes use of a prejudice function m(X) being normalized, i.e. Jm(X)d 3 X=1, where d 3 means 
that the computation is done in 3D (in a 2D reconstruction application d 2 will be adequately used). The normalized 
prejudice function can be inserted externally from an inserting device 21 or could be computed by the computer 12, 
which for instance could make use of the estimated reconstruction to provide the initial prejudice function to start with. 

10 The computer 1 2 computes the high fidelity reconstruction using several iteration cycles. The prejudice function is also 
recomputed in each iteration cycle. 

[0063] The initial prejudice distribution function provides a vague prejudice distribution of the reconstruction of the 
specimen 1 . It is to be noted that a high probability of density value somewhere in the reconstructed sample indicates 
that it is a high probability that something really could be found there. A low probability of density value in the recon- 

15 structed sample thus indicates that it is a low probability that something really could be found there. Thus, the prejudice 
distribution function is used to provide a reasonable result from shifts computed of reduced chi-squared statistic for 
each iteration. The reduced chi-squared statistic should asymptotically reach +1 , and the entropy maximise d for each 
iteration. Both are functions dependent on m(X) and of the next preceding iteration reconstruction result. 
[0064] The original prejudice function could be rather vague and freely chosen. However, the proability distribution 

20 function m(X) is to be recalculated (and normalized) for each iteration in order to sharpen it. This could for instance 
be done by using the updated density distribution in every iteration, the normalization and blurring of it becoming the 
new prejudice distribution function for the density in the new reconstruction. 

[0065] Principally, a blurring could be done by making a Fourier transform of the estimated structure and by multiplying 
the coefficients of this Fourier transform with the Fourier coefficients of a gaussian and spherical filter. The reason to 
25 make these transformations instead of working in the real space is that the coefficients of the filter will be so small that 
subpixel problems arise. The problem is more severe the lesser the blur is, which means that this problem gets more 
severe as the iterations proceed. 

[0066] The blurred function so received has a decreased resolution and is, after normalization, the new prejudice 
distribution function for the new iteration. 
30 [0067] The blurring could be rather strong at the first blurring operation, for a resolution of for instance 100 A even 
if the observed data has an expected resolution of for instance 50 A. Then the blurring is decreased for each iteration. 
There is principally no lower limit other than that the blur must be higher than zero. 

[0068] The prejudice function has the purpose of providing a faster convergence to the final reconstruction. A problem 
if only the reduced chi-squared statistic is used is that noise will have a great influence, and this causes such an iteration 
35 process to stand still after a while and thus not be completed. The correction according the invention using the prejudice 
distribution allows the iterations to proceed. 

[0069] Thus, in each iteration cycle a next, refined structure will be calculated, and a next, refined prejudice distri- 
bution will be generated. The next cycle prejudice distribution, is the refined structure of the current cycle, blurred with 
a gaussian and spherical filter and normalized. The next cycle prejudice distribution could also be generated in other 
40 ways, but the further the iterations proceed, the more accurate the prejudice distribution must be. This continuous 
improvement of the prejudice distribution results in a much quicker convergence of the iterations than would be the 
case with only the conditions on the reduced chi-squared statistic, which is too susceptible to noise. The size of the 
calculations is hereby decreased, and the resolution is improved. 

[0070] According to the invention, a constrained maximum entropy tomographic reconstruction is computed in real 
45 space, which exploits the stored data of the estimated reconstruction and producing a 3D structure whose projection 
convoluted with the point spread function of the microscope will adapt to the observed structure, to give a low-noise 
3D reconstruction that is quantitatively correct. The output of the computer is fed to the presentation device 16 pre- 
senting the reconstructed data as a 3D image on a visual display. 

[0071] Given that F(X) represents the density values at points (x,y,z) in the 3D reconstruction, and that m(X) is the 
50 initial prejudice distribution, normalized such that J m(X)d 3 X=1 , then the constrained maximum entropy principle implies 
that F(X) must be determined 

a) to maximise the entropy 

55 _ » 

S(F)=- |F(X)ln(F(X)/m(X))d 3 X (2) 
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b) to maintain the normalization of F(X) so that jF(X)d 3 X=1 

c) and to aim the reduced chi-squared statistic towards +1 under the constraint of prior knowledge or prior prejudice 
distribution m(X) being the result of the next preceding calculated distribution blurred with a gaussian and spherical 
filtering operation. The constraint on the reduced chi-squared statistic ensures that the reconstructed density is 
consistent - but not over-consistent - with its observed projections. Entropy maximisation ensures that the density 
modulations are as conservative as possible, giving the smoothest achievable density by eliminating details which 
are not significantly above noise and so flattening any artefacts that arise from systematic errors. 

[0072] Thus, a harmonisation with data must be improved and therefore the reduced chi-squared statistic must be 
driven towards +1 at the same time, because the reduced chi-squared statistic is a measure of how the measured and 
calculated data differ from each other. The result is that each iteration gives a reconstruction which is as blurred as 
possible but which represents the recorded data about the specimen in a more confident way than what has been 
provided before with conventional reconstruction methods. 

[0073] However, the blur of the reconstruction decreases for each iteration cycle so that the final result a) will explain 
the data in an optimal way but b) will contain as little information as possible in order to prevent overinterpretations. 
[0074] The 3D density distribution is built on the 3D grid 22 inserted in the computer, with the understanding that the 
grid values define a continuous function through trilinear interpolation. This is a convolution with the function: 



20 



25 



Wa(x)* WKy)* W 7 (z) , where Wa(x) = 



lM for\x\<a 
a 

0 for\x\^a 



(3) 



30 



R*.*,(X)= £ 2 £F(na,ip,mY>* W«(x - na>* W P (y - lp> - nry) (4) 



35 The function W a (X) is used to denote the interpolation function in 3D. Let P0> denote the projection operation to the i 
th view; then the contribution at (x< i ),y(0)=(xO)) on the i:th tilt of grid value F(X) at grid point X in the 3D grid will be: 



40 



45 



And thus the i:th projected view F< i )(x( | )) is given by 
FP>( 5 (i>) „ £ F (x)*{p (i) W 8 (^ -P P) X)} 



(5) 



(6) 



[0075] Each projected image is also subjected to convolution (denoted by ®) by the point spread function PSF: 
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55 



[0076] Combining (6) and (7) we get 



(7) 
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5 



(8) 



[0077] Introducing the smearing function, the projected PSF, 



10 



T^x.xra) = ((p (i) w«) • psf^x™ - p (i) x) 



(9) 



we can finally write for the projection 



15 



^( 5f(5) ) = S F (x)*^ l >(X,xP') 

X 



(10) 



20 



[0078] in Fourier space, the PSF is the product of the pupil function and the CTF. Both the pupil and the CTF functions 
can change over the tilts (when recordings of a specimen is made using an electron microscope), since, in principle, 
each tilt can be recorded at a different focus with a different setting for the lenses in the electron microscope 1 0. For 

25 a given tilt, the PSF is only similar for points in a section, which is perpendicular to the beam direction, of the 3D density. 
However, if the specimen is not too thick, i.e. dynamic and multiple scattering can be neglected, the PSF is similar for 
all points in the 3D density. From the point of view of numerical calculation of the projection, an efficient implementation 
is to make use of this property for the projected PSF, i.e. creating a smearing function to be explained further below. 
Consequently the smearing function which can be accurately estimated with a small matrix need be computed for only 

30 one 3D point. The accurate projection is subsequently calculated by convoluting the projected density from its grid 
points with the smearing function. Tilted views have a focus gradient across the image and thus a varying PSF function 
of the 2D image. The projection can then be computed using a set of precomputed PSFs in the convolution calculation 
with the projected density. When focus variation must be considered for thicker specimens, the 3D density can be split 
up in layers of appropriate thickness, and each layer convoluted with its proper PSF. Subsequently, the summed density 

35 from all the layers gives the total projection. The densities in each projected images are given by the equation (1) above. 
[0079] The iterative solution to the constraint maximum reconstruction can be formulated as 



where F new (X) is the new calculated reconstruction, F old (X) is the next preceding calculated reconstruction, u(X) is a 
function built from the gradient of the entropy (see equation 16), v(X) is a function built from the gradients of the chi- 
squared statistic (see equation 17), and a and p are scalars described further below. The updated map F new (X) is used 
in the next iteration cycle both as the density extimate and to generate the prior prejudice map. 
45 [0080] The equation (11) shows how each estimated density distribution is updated to the next iteration cycle. The 
calculation is such that two features are provided at the same time, firstly maximising the entropy and secondly driving 
the reduced chi-squared statistic towards +1 . If only the entropy were maximise d then only a blurred calculation would 
be provided. 

50 THE CHI-SQUARED STATISTIC 

[0081] In each iteration cycle F^'^pa^xO)) is compared to F<') obs (x(')), scaled in the operation 14, for the i:th pro- 
jection according equation (1). Its chi-squared statistic C<') is calculated: 



F new (X)=F o|d (X)+a*U<X)+fJ*v(X) 



(11) 
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(12) 
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where 



•a 2 (x pr 



5 

is the variance of the observed data for the i* h projection at point x. This gives as total chi-squared statistic 

10 



c= £c« (13) 



t5 [0082] The values 

should not be fitted better than their expected variances. Consequently, at the situation of optimal fit we get 
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30 



35 



C = £ (Number of gridpoints) 03 ( 14) 

[0083] A handy measure for monitoring of the chi-squared statistic is therefore the reduced chi-squared statistic 
Q<i) _ J. opaaalSt , j qq 

1 (Number of gridpoints) (i) * • (15) 

The iterative formulation 



[0084] The construction of the 3D density F(X) which maximises the entropy S in equation 2 while satisfying the 
condition (1 4) on the constraint function C proceeds iteratively from an initial estimate. At each iteration F(X ) is modified 
40 by a linear combination of two search direction u(X) and v(X), chosen as the normalized gradients of S and C according 
to: 

u(X)=v£s (16) 

45 

v(X) = V£c (17) 

[0085] Let a and P denote the coefficients of u(X) and v(X) in the linear combination giving the correction in F(X ) at 
so the current iteration. We then approximate the variations of S and C in the 2D search subspace spanned by u(X) and 
v(X) by quadratic models: 



55 
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C(a,P) = C(0,0)+a»(u(x)|v ± C(P,0))+^(v(x)|v jl C(0,0))+ 



Mm 



H(C)u(x)} (u(x) 
H(C)u(xj) (v(x) 



H(C)v( X )}J 



x)\ UJ 



(18) 



S(a,£) = S(0,0) -i- a*(u(x)| V s S(0,0))+ ^(vfxJlv^O.O)) + 



2UJ(v(x) 



H(S)u(x)) <u(x) 
H(S) U (X)) (v(x) 



H(S)v(x))YcA 
H(S)v(x)) p) 



(19) 



[0086] The first order derivatives of C and S with respect to a and 0 are: 



= (u^^C^O)}- £v£S(0,0). V^iCO) 



(20) 



= (v^V^O)) - J V^C(O.O). V.C(0,0) 



(21) 



= (u(x)|v t S(0,0)) - J V^S(0,0) * ^(0,0) 



(22) 



^ - (v(x)|v t S(0,0)) - J V«Q0,0) • V^O.O) 



(23) 



[0087] The second order derivatives (the elements of the Hessian matrices in (18) and (19)) of C with respect to a 
and p are: 



-£_<L 



-^•«X)|H,C H X)).T ? ^0<..(X.^.)^1 (24) 



[0088] The function 0(')(X , Y ,x(')) is an overlap function, is>0 and is defined by the smearing function (see equation 9) 



12 





EP 0 885 430 B1 




(x.Y.x«>) - T (i) (x t x< 5 >)«gT< l »(Y,y«) 



(25) 



5 



[0089] However, the smearing function only has a significant value at a limited number of grid points. Thus, it is 
convenient to write 



15 remembering that for each coordinate on I th tilt, the list of overlap values > 0 is small. 

[0090] The time needed for calculating expression (24) must be reduced before an attempt to evaluate it is made. 
Even for a small case with 1 0 2 tilts, 1 0 4 points per tilt and 1 0 6 gridpoints in the 3D volume, the expression in (24) implies 
the evaluation of the overlap function (25) at 10 18 points. 

[0091] For a fast computer with a capacity of 1 0 8 expressions evaluations per second, it would still take approximately 
20 300 years to evaluate the formula. The size of the calculation can be reduced by making use of the reasonably good 
approximation that the smearing function is similar for all the different 3D grid points (at the least, it is similar for those 
points in layers perpendicular to the beam direction, see the discussion following equation (10) 



30 From (27) we see that these Hessian matrix elements (the elements of the Hessian matrices in (18) and (19)) can be 
readily computed by projecting the entropy and the chi-square gradients and then multiplying the projections through 
a limited convolution with the overlap function. Instead of 300 years of evaluation time, this expression can be evaluated 
in about 1 minute, a speed-up of 10 8 times! 

[0092] The second order derivatives of S with respect to a and p are:- 
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(26) 



25 




35 




(28) 
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(29) 



50 




(30) 



[0093] 



The gradients V^C and V X S are calculated in the following way 
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mCY)J 



^F(X) 



dY 



[m(X)j 



(31) 



JO 



(32) 



[0094] Using the chain rule, we can write 



15 



20 



(33) 



[0095] From equation (1 2) we see that 



25 



(34) 



30 and from equation (10) we get 



35 



(35) 



40 



[0096] Thus equation (33) can be written as 



45 



(36) 



50 



55 



With equation (36), it can be seen that all physical and numerical parameters needed to compute equations (18) and 
(19) are known, and the calculation is feasible. The iterative solution to the constrained maximum entropy tomography 
reconstruction in equation (11) is hereby completed. 

[0097] Thus, as seen in the computer block 12 in FIG 1, in accordance with the iteration equation (11) above, the 
output F new from the calculation means 15 is fed to_a normalization operation 22 which feeds F old as an input to 15. 
u new (X) is normalized in an operation 23 and v new (X) in an operation 24. The outputs of 23 and 24 are provided as 
inputs to the calculation means 15 together with their respective scalars. The new m(X) is also normalized in an op- 
eration 25 which feeds it as an input to the calculation means 15. 

[0098] The scalars I and J can be determined by solving a generalized eigenvalue problem in a wellknown way, for 
instance the way described by G. Bricogne, Acta Crystallographica A40, pp 410-445 (1984). The updated map F new 
(X) is used in the next iteration cycle both as the density estimate and to generate the prior prejudice map. 
[0099] The constraint on reduced chi-squared statistics ensures that the reconstructed density is consistent, but not 
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over-consistent with its observed projections. 

[0100] The calculations in the computer are illustrated in the flow chart in FIG 2A to 2C and will now be further 
described. The program is preferably divided into three different subprograms. 

First subprogram in FIG 2A 

[0101] 

51. Loading the first estimate of the density distribution from the unit 19 in FIG 1. 

52. Normalization of the estimate F(X ) so that \F(X)dX= 1 This is used as the F old (X ), a prior density distribution that 
is used as a starting value for the iteration to be done, in which the equation (11) is to be calculated. 

53. Loading a prior prejudice (initial knowlege) distribution m(X ), or generating it by blurring the estimate distribution 
15 with a gaussian and a spherical filter, and normalizing m(X). 

54. Insertion of parameters, such as projection scale, variances, guide functions etc. 
Insertion of the number of grid points. 

20 S5. Reference data for improving the scaling is calculated in each iteration cycle in order to monitor and eliminate 
"series termination errors" during limited Fast Fourier Transform calculation. This is a condition for the scaling 
done by the third subprogram in FIG 2C. The accuracy is needed so that the scaling is accurate enough to make 
the measured differences relevant. 

25 Second subprogram in FIG 2B (generates the projections) 

[0102] 

56. Loading prior density distribution from S2. 

57. Loading geometry data describing the projection geometry. 

58. Loading individual parameters for each projection defining how they were taken, for instance acceleration volt- 
age, astigmatism, lens assymetry, focus, chromatic and/or spherical abberation or the like. 

59. Is this the first iteration? If yes then go to S10, if no then go to S11. 



30 



35 



510. Calculation of overlap functions (O), "smearing functions" and recognising that they have significant values at 
a limited number of grid points. Calculating reference data not coming from the observations of the sample, i. 

40 e. calculation of projections with eliminated data. 

51 1 . Calculation of reference data in order to eliminate "series termination errors" by the limited Fast Fourier Trans- 
form calculation at applications of PSF. 

45 S1 2. Loading the precalculated PSF (point spread function) from the unit 1 7 in FIG 1 (or loading optical parameters 
derived from measurements of the microscope 10 and calculating its PSF). Loading the calculated smearing 
functions and reference data. 

S1 3. Calculation of projections. Correcting every projection by the optical parameters defining the PSF. Determining 
50 the number of 2D projections to process. 

Third subprogram in FIG 2C (the 3D calculation) 

[0103] 



55 



S14. First a lot of supporting data and data calculated in the other subprograms are inserted, such as: 
Conditions for the iterations to start and to stop for presenting the provided result. 
Loading the observed projections. 
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The sorted projections provided in point S13. 

The variances for each grid point. These variances are either measured or modelled since it is a Poisson 
distribution. 
Size data. 
5 Series termination errors. 

Geometry description in order to make projections in right directions. 

The image parameters. 

The number of 2D projections etc 

10 [0104] The essential features are however that the observed and calculated projections and the variances are in- 
serted. 

515. Scaling in order to denormalize calculated projections against observed data in order to provide a common 
level and to be aware of how an absolute level is to be provided. A certain electron dose for the specimen 

15 corresponds to a certain grey scale giving for instance molecular weight. 

516. Calculating the reduced chi-squared statistic and goodness of fit parameters (sums of quadrated deviations) 
by comparing the scaled, calculated projections to the observed data for monitoring progress from one iteration 
cycle to the next one. 

20 

51 7. Approximating reduced chi-squared statistics and entropy by quadratic models close to the current trial solution 
F(X) using the equations (16) and (17). 

51 8. Calculating the total entropy of the density estimate and total reduced chi-squared statistic for scaling purposes. 
25 The first order derivatives of C and S with respect to a and p are derived from the equations (20) to (23). 

Calculating gradients of the entropy and the reduced chi-squared statistic as well as the other derivatives (Hes- 
sian matrix elements) from the equations (27) for the reduced chi-squared statistic and equations (28) to (30) 
for the entropy by projecting the 3D gradients of the entropy and the reduced chi-squared statistic (equation 27). 
All these gradients and the overlap function calculated in step S12 are needed for the quadratic model and are 
30 used in the equations (16) and (17). 

519. Solving the coefficients a and p of u(X) and v(X). Updating the density with the parameters according to the 
equation (11 ). Determining a and p from the point where the entropy is maximise d and the reduced chi-squared 
statistic is driven towards +1 using said parameters. Calculating F new (X) in equation (11). 

35 

S20 If a new iteration is to be made (conditions for iterations could be that they are done in a predetermined number 
of cycles or that they are done to a prescribed correlation limit or until the entropy maximation will be weakened) 
go to S2 else provide the completed reconstruction on the input of the presentation device 16 in FIG 1. 

40 [01 05] Thus, when the calculations described above are completed, the 3D image is reconstructed from the iteratively 
calculated data, and the reconstruction data is fed to the visual display means of the presentation device 16, where it 
provides a visible image of an object, which, when having no symmetry, could not be visible by any other known means. 
[0106] Computing the second order derivatives uses, among others, a function 0(i)(X,Y,x(i)), which is an overlap 
function defined by the smearing function T(i)(X ,x (i)), but the smearing function is a measure of how much the density 

45 value in each point is influenced by the other points. In fact, only a limited number of neighbouring points influence 
each point, and thus, the smearing function only has a significant value at a limited number of grid points. This reduces 
the size of the calculations. 

[01 07] With a method according to the invention, the known equations are parametrisized to a level where the physical 
meaning of each component can be identified/This means that the right values can be assigned and intelligent sug- 
50 gestions can be made, which in turn makes the equations possible to solve. 

[0108] The difference between projected data and observed data is also minimised. (32) and (33) to (35) relate the 
gradient error in 3D to the error in 2D and if used together with the second order derivatives makes it possible to solve 
(15) and (16), which give the variation of the total C and S and thereby enable maximising S and minimising C towards 
+ 1. 

55 [01 09] Figs 3 A to 3D illustrate how the feasibility and correctness of the reconstruction made according to the inven- 
tion has been tested by reconstructing the unstained and vitrified adenovirus from electron micrographs and compared 
to projections of the observed data and to projections provided with another 3D reconstruction method. 
[01 1 0] Figs 3A is observed data show of 2D-projections recorded by an electron microscope of an adenovirus particle, 
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which is icosahedral with a maximum diameter of 1400 A, including twelve fibres protruding from twelve vertices. 
Without the fibres, the virus can be contained in a sphere with a diameter of 914 A. 

[0111] Fig. 3B shows a reconstruction of the adenovirus in 3D to approximately 35 A resolution, made by aid of a 
CL method described by Crowther, R. A. in 'Philosophical Transactions of the Royal Society of London' B261, 

5 pp221-230 (1971) and modified and improved by S.D. Fuller as described in Cell 48, pp 923-934 (1987). The CL 
method is of another kind than the inventive method. According to the CL method a brief 3D image is reconstructed 
and is thus not as refined as the inventive method and lacks among other things the prior prejudice approach. A 
refinement of the reconstruction made by the CL method could then be provided in several conventional ways. The 
particle is contained in a cube with sides of 135 pixels and is projected onto a 150x150 pixel canvas. 

10 [0112] Fig 3C is projections from a reconstruction made according to the invention. The projection geometry in FIG 
3C is the same as in FIG 3B. The background is so low in Fig 3C that it is hard to distinguish the projected background 
of the virus inside the cube from the canvas with no projected values. 

[0113] As mentioned above when disclosing the elements 10, 11 and 18in Fig. 1 the CTF (Contrast Transfer Function) 
could be supplied as an experimental curve or as parameters refined from a parametrized model. In accordance with 

15 the numerical recipes following Eq. 10 above, a compensation also could be provided for a CTF due to the focus 
gradient in tilted views or thick specimens. Because this entails a more complicated summation scheme for the projected 
PSF function (defined in Eq. (9)), the evaluation of the Eqs. 10, 25, 27 and 35 is also correspondingly more elaborated 
in the inventive implementation. In the case of a constant PSF, (Point Spaced Function) the evaluation is a straight- 
forward evaluation as indicated by the formulas. In the example described above, the possibility of a varying PSF 

20 function was not applied since the micrographs of the specimen, adenovirus, were recorded with nearly the same 
defocus and from untilted views, and the focus variation across the virus in the beam direction was neglected. 
[0114] It is also within the scope of invention to have a sample to be reconstructed to be an outcut envelope from 
an object to be observed. The rest of the object is then a buffer which can be reproduced from iteration to iteration 
without reconstruction, i.e. unchanged. Normally, the iteration work is done when calculations are made on a field. 

25 However, when reconstruction is to be done on for instance an icosaedric virus the interesting part of it could perhaps 
only be its shell. Then, the program in the reconstruction device will provice an envelope at the place of the shell to be 
reconstructed and image the rest of the virus as some kind of noice level. If a symmetric body is to be reconstructed- 
then only one of the symmetric parts could be reconstructed and the program coult then join the symmetric parts 
reconstructed after that the reconstruction of the reconstructed part is done. 

30 [011 5] Expansions of the same concept could also be made is a situation when some of the parts of the concept are 
previously known. Assume that a reconstruction should be done of a big molecular complex. At least one of the mol- 
ecules in the complex which place in the complex is known could be found in a manual or the like. Its atomic structure 
at high resolution happens to be well known. This density distribution could thus be generated and inserted in the 
reconstruction system. That part is then cut out from the reconstruction, i.e. an envelope is done of the other parts 

35 excluding the well known part. The reconstruction of the complicated structure could then be done using fewer variables 
and fewer degrees of free-dom than if the reconstruction was made on the whole structure and with the same result. 
This implies that the numerical force could be enhanced for the rest of the structure. 

[0116] Thus, it is possible to insert known fragments in the object to be reconstructed and thus enhance the resolution 
of the remaining parts. For instance at a tomographic reconstruction of a patient having injured tissue but for instance 
40 the bones are intact then the bones need not be reconstructed and thus only the tissue. 

DESCRIPTION OF A SECOND EMBODIMENT 

[0117] FIG. 4 shows a second embodiment of an apparatus for data acquisition and processing according to the 
45 invention, used in order to achieve a 2D image reconstruction at high resolution of a picture which is presented for 
instance on a television monitor or on paper or as an oil-painting or the like. In the 3D reconstruction described above 
the projection data was provided as 2D projections. In the 2D reconstruction in this embodiment the "projections" are 
provided as lines. 

[01 18] Elements corresponding with elements in FIG. 1 have the same reference numerals. It is to be noted that the 
50 calculation device preferably is a fast computer being able to process images at least with the same speed as the 
vertical rate of a TV-monitor. 

[01 1 9] An imaging and scanning unit 40 is connected to an input of a filtering and normalization means 43 comprised 
in the computer 12 . The imaging and scanning unit is scanning a picture or a TV screen on which a picture is presented. 
When scanning a picture on a TV screen several scanning sequences are done for each picture in order to have an 
55 overdetermination of it, since the meaning of providing this inventive kind of operation is to enhance the image reso- 
lution. The output of said filtering and normalization means 43 is connected to an input of the calculation means 15 
also comprised in the computer 12. The imaging and scanning unit 40 is also connected to a point spread function 
generator 42, the outputs of said function generators being connected to the computer 12. An output of the calculation 
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unit 15 is connected to a presentation means 46 which in this embodiment when processing TV picures for instance 
could be a large sized high resolution screen or the like. 

[01 20] The computer 1 2 is supplied with digitized image data from a unit 44 for providing digitized data of an estimated 
structure. The unit 44 could comprise the television receiver unit itself or a video camera directed to the picture to be 

5 processed and an A/D converter. However, almost any picture scanning data could be presented as the estimated 
structure but the computing will be faster if the resemblance beween the picture to be processed and the estimated 
picture is good. The output of the unit 44 is normalized before it is fed to the calculating means 15 in the computer 12. 
[0121] A prior prejudice distribution m(X) is either inserted separately to the calculating means in the computer 15 
by a unit 44 or created by the calculating means 15 itself by blurring the estimated structure by making a Fourier 

10 transform and multiplying the coefficients of this Fourier transform with the Fourier coefficients of a gaussian and 
spherical filter. The prior prejudice distribution is normalized. 

[0122] Since the resolution in the processed picture is to be enhanced a rather dense 2D grid 47 is fed into the 
computer 12. 

[0123] The computation will be done according to the same principles as described above but in a less complicated 
15 way. 

[0124] The data is processed in accordance with the inventive method, so that a higher resolution and a lower noise 
level is achieved than is possible using conventional techniques. The processed data is then fed to the presentation 
means 16 where it provides an image of greatly improved quality. 

20 DESCRIPTION OF A THIRD EMBODIMENT 

[0125] This embodiment is a card reading unit for checking pass-cards and processes data in 3D form. It relates to 
a kind of pattern reconstruction. A card 50 having a number of small designs 51 being slightly different but which for 
the naked eye seems to be the same is inserted in a card reader 52. The designs could illustrate almost anything, such 

25 as stars, rings, clouds, trees, bears. The designs 51 have the same positions in all cards adapted to be read by the 
card reader 52 having either an individual scanning device per design or having a movable scanning device scanning 
the designs in sequence. The scanned designs could be regarded as different 2D projections of a 3D sample and are 
presented to the filtering means 13 in the computer 12. However, the filtering process could be extensively simplified 
as compared to what is necessary in the embodiment according to FIG 1, since the card reader could be designed 

30 such that there is no need for processing regarding geometry description and bringing the projections in harmony with 
each other. 

[0126] Parameters for providing a PSF 53 of the card reader 52 is fed to the calculating means 15 in the computer 
12. The computer is a part of the card reading unit and this means that these parameters could be presented to the 
computer during manufacture and the PSF could be calculated and stored at that time. 
35 [0127] The digitized data of estimated structure 54 could also be provided and normalized during manufacture for 
instance comprising designs of the same structure as on the card but for instance all being exactly alike or else being 
read from a reference card provided in the card reader in the factory and normalized and stored permanently. 
[0128] The prior prejudice distribution m(X) 55 could also be normalized and stored premanently at the factory and 
also the 3D grid 56. 

40 [0129] The computer processes the data in the way described above and send the result to a unit 57 when the last 
iteration is done. The card 50 could have a conventional magnetic strip 58 or the like providing a simple identification 
of the card. Alternatively, the card owner could have a personal code or the like inserted through an identical data 
insertion means which could be a keyboard, a voice identifier or the like. 

[0130] When a card 50 is inserted into the card reader 52 the identity of the card is read from the strip 58 and provided 
45 directly to the calculating means 1 5 which in turn feeds it to the unit 57. The unit 57 cooperates with a storage 60 having 
processed data of cards stored in addresses identified by the identity of the card. 

[0131] If the inserted card has been used earlier the data for it is found and delivered to the unit 57. When the 
processing operation of the designs on the card is ready the unit 57 compares the stored processed data with the new 
one and accepts a card only for a true compare. 
so [01 32] If the inserted card has not been used before and thus data of it is not stored then an identity check could for 
instance be provided in an external register through an external connection device 61. 

[01 33] This device could also receive updated information about cards from a central office. The identity cards could 
be used in offices to provide passage to different regions in a building or the like. In this connection, there is frequently 
a need for registration and deregistration of cards. 
55 [01 34] If data is found in the external register with a register mark that it is allowed for the card reading unit in question 
then it could be transported to the unit 57 which stores the data in the storage 60 and also makes the checking com- 
parison with the newly processed data. 

[0135] Another possibility is that when a new card is issued and handed over to a card owner then registration data 
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such as the information on the strip 58 is sent over to the unit 57 through the device 6 1 . The unit 57 stores the registration 
data. The first time the card is inserted in the card reader and the data from the designs 51 are calculated by the 
computer the data is stored in the storage 60 and the card owner can pass. 

[0136] Still another possibility is to have the data from a lot of cards stored in the storage 60 in dummy addresses 
and to change the address to an ordinary one by distant control as soon as a card is handled over to a new owner. 
[0137] If a deregistration information for a card is fed to the device 61 then the unit 57 could either put a note in the 
store registration that the card is invalid or erase the data. 

[0138] The teachings of the inventions is also adaptable on 1 D data reconstruction, such as on several temperature 
measurements in one occation or the like. In such a case an input device connected to the computer 12 records variation 
of the quantity in relation to a predetermined variable and the calculation means reconstructs the measured quantity 
as a series of measurement data. An application on telecommunication is also supported by the invention, for instance 
by making an oversampling on each sampled analog signal. Even though 3D imaging of smaller objects is described 
in the first embodiment 3D imaging of larger objects, for instance tomographic recordings, such as PET scanning (PET 
= Position Emission Tomography), is supported by the invention. The invention is also applicable on holography both 
on thin film and volume holograms. Other applications relates to nuclear medicine, X-ray diagnostic imaging, such as 
mamography and odontologic imaging, ultra sound imaging, radar imaging, IR-imaging, meteorological charts, geo- 
graphical sharts, CT scanning etc. 



1. A method for providing high fidelity reconstruction built on a grid (122;47;56) having grid points of an observed 
sample (1;40;51), said method comprising the following steps: 

a) providing several recorded, observed data of the sample (10,11 ;40;52), each from a different aspect of the 
sample; 

b) providing a variance (S14) for individual observation grid points in each recorded observed data; 

c) providing a point spread function (PSF) (18;42;53) dependent on the properties of the recording device; 

d) providing an initial, freely chosen, estimated density distribution of the sample (19;44;54) at the individual 
grid points; 

e) providing a blurred prior prejudice density distribution at the individual grid points using the initial estimated 
distribution, the variance and the point spread function; 

f) calculating (12) in an iterative process for each iteration: 

* a new estimated distribution of the sample at the grid points using the prior prejudice density distribution 
and comparison between the estimated distribution in the next preceding iteration and the recorded, ob- 
served data of the sample, 

* a new prior prejudice density distribution at the individual grid points on the new estimated distribution 
less blurred regarding the density distribution at the grid points, using the variance and the point spread 
function, than the prior prejudice distribution in the next preceding iteration; 



2. A method according to claim 1 , in which blurring is provided by making a Fourier transform of the estimated structure 
and by multiplying the coefficients of this Fourier transform with the Fourier coefficients of a gaussian and spherical 



3. A method according to claim 1 or 2, characterized by normalizing said blurred structure before it is used as the 
new prior prejudice distribution in the next iteration cycle. 

4. A method according to any one of the preceding claims, characterized by outcutting an envelope from an object 
to be the sample to be reconstructed; providing the rest of the object as a buffer to be just reproduced from iteration 
to iteration without reconstruction. 

5. A method according to any one of the preceding claims comprising the following steps: 



Claims 



continuing the iterations until the difference between the new estimated distribution and the next preceding 
estimated distribution is less than a predetermined density resolution distribution condition on the grid 
points, and 

providing the latest iterated density distribution as the reconstruction. 



filter. 
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• for each iteration in order to provide a faster convergence to the final reconstruction: 

calculating a reduced chi-squared statistic (C)as a measure of how the measured and calculated data 
differ from each other using the immediately preceding calculated estimated distribution, the recorded 
5 observations and said variance, and the grid, while maintaining normalization of the calculated density 

distribution, 

calculating an entropy (S) using the immediately preceding calculated estimated density distribution and 
the new prior prejudice distribution; 

10 • maximising the entropy for providing the smoothest achievable density under the constraint of driving the 

reduced chi-squared statistic towards +1, and providing the reduced chi-squared statistic and the entropy for 
the calculation of the new calculated estimated density distribution to use in the next iteration cycle. 

6. A method according to anyone of the preceding claims, further comprising during the calculation of the new prior 
is prejudice density distribution: 

providing each new estimated distribution with a convolution of the point spread function; 
introducing a projected point spread function, called smearing function, on the calculated density distribution; 
approximating that the smearing function is greater than zero only for a limited number of grid points of said 
20 sample; and 

producing a structure whose projection convoluted with the point spread function will adapt to the structure of 
the observed data. 

A method according to Claim 6, further comprising: 

providing an overlap function defined by the smearing function, 
driving said reduced chi-squared statistic towards +1, and 
maximising the entropy, using said overlap function (S18). 

30 8. A method according to claim 7, wherein the several recorded observation data of the sample are provided by the 
recording means (10,11 ) supplying 2D projections in digitized form of the sample and forwarding the data contained 
in them to the calculation means (12) in order to provide a 3D image reconstruction, characterized by processing 
said data with the use of an approximation that the smearing function is greater than zero only for a limited number 
of the grid points (122) in a layer perpendicular to the beam direction, and by producing the 3D structure whose 

35 projection convoluted with the point spread function will adapt to the observed structure (Equation 7). 

9. A method according to anyone of the preceding claims for a 3D reconstruction of a sample, characterized in that 
said observations of the sample are a series of differently tilted 2D observations of the sample. 

40 10. A method according to anyone or the claims 1 to 8 for a 3D reconstruction of a sample by means of light recordings, 
characterized in that said observations from different aspects of the sample are a series of 2D observations 
focused at different depths of the sample. 

1 1. A method according to anyone of preceding claims, characterized in that the reconstruction is provided of organic 
45 objects, for instance small organic objects, such as virus particles or the like. 

12. A method according to anyone of the claims 1 to 8, characterized in that the reconstruction is a pattern recon- 
struction of sample elements (51) having slightly different designs provided on a card (50) in order to provide a 
secure and unique identity of the card. 

50 

13. A method according to anyone of the claims 1 to 6 for a 2D or 1 D reconstruction, characterized In that the different 
aspects of the sample are lines and dots, respectively, across the sample. 

14. An apparatus for providing high fidelity reconstruction of an observed sample of an object built on a grid having 
55 determined grid points, said apparatus comprising: 

input means (10,11;40;52) providing observed data of the sample; 

means for providing a variance for individual observation grid points in each recorded observation; 
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point spread function(PSF)insertion means (18;42;53) to insert a point spread function (PSF) dependent on 
the properties of the recording device; 

estimated distribution means (19;44;54) providing an initial estimated, freely chosen density distribution of the 
sample on the individual grid points; 

prejudice distribution means (21,25;45;54) providing a blurred prior prejudice density distribution on the indi- 
vidual grid points using data the estimated distribution means, the variance and the point spread function; 
calculating means (12), for calculating in an iterative process for each iteration: 

a new estimated density distribution of the sample on the grid points using the prior prejudice density 
distribution and comparison between the estimated density distribution in the next preceding iteration and 
said observed data of the sample from said input means, 
* a new prior prejudice density distribution on the individual grid points on the new estimate less blurred 
regarding the density distribution at the grid points, using the variance and the point spread function, than 
the prior prejudice distribution in the next preceding iteration; 

said calculation means (12) further comprising means to continue the iterations until the difference between 
the new estimated density distribution and the next preceding estimated density distribution is less than a 
predetermined condition; and 

means (16;46;57) connected to said calculation means for presenting the calculated result as the reconstruc- 
tion. 

15. An apparatus according to claim 14, in which said input means (10,11) measures several aspects of a quantity 
(1), characterized by 

said calculation means (12) for each iteration cycle in order to provide a faster convergence to the final reconstruc- 
tion comprising means to carry out the following calculations: 

a reduced chi-squared statistic (C)as a measure of how the measured and calculated data differ from each 
other using the immediately preceding calculated estimated distribution, the recorded observations and said 
variance, and the grid, while maintaining normalization of the calculated density distribution, 
an entropy (S) using the immediately preceding calculated estimated density distribution and the new prior 
prejudice distribution; 

maximising the entropy for providing the smoothest achievable density under the constraint of driving the reduced 
chi-squared statistic towards +1 , and providing the reduced chi-squared statistic and the entropy for the calculation 
of the new calculated estimated density distribution to use in the next iteration cycle. 

16. An apparatus according to claim 15, characterized in that said calculation means (12) comprising means to 
calculate a projected point spread function, called a smearing function, being greater than zero only for a limited 
number of grid points of said sample and producing a structure whose projections convoluted with the point spread 
function will adapt to the observed structure. 

17. An apparatus according to claim 16, wherein the input means comprises a device (10;52) supplying observed data 
of 2D projections in digitized form of the observed sample and forwarding the data to the calculation means in 
order to provide a high fidelity 3D image reconstruction, characterized in that said calculation means (12) further 
comprising means to process the data with the use of an approximation that said smear function is greater than 
zero only for a limited number of grid points in a layer perpendicular to the beam direction and producing the 3D 
structure whose projections convoluted with the point spread function will adapt to the observed structure. 

18. An apparatus according to anyone of the claims 14 to 17, characterized in that said input means is a microscope 
(10), for instance an electron microscope, a light microscope, or a confocal microscope. 

19. An apparatus according to anyone of the claims 14 to 17, characterized in that said input means is a card reader 
(52) means to read a pattern of sample elements (51) having slightly different designs provided on a card (50) in 
order to provide a pattern reconstruction for providing a secure and unique identity of the card. 

20. An apparatus according to anyone of the claims 14 to 17 for a 2D reconstruction, characterized in that said input 
means is a scanning device (40) to scann a 2D image; and that said projections are lines across the sample. 
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21. An apparatus according to anyone of the claims 14 to 17, characterized in that the input means (52) comprising 
means to record variation of the quantity in relation to a predetermined variable and said calculation means com- 
prising means reconstruct said measured quantity as a series of measurement data. 

5 22. An apparatus according to anyone of the claims 14 to 21 , characterized in that the estimated and the predjudice 
distribution means (21 , 25; 45; 54) comprising means to work on an outcut envelope from an object to be observed, 
the envelope being the sample to be reconstructed; and that the estimated and the predjudice distribution means 
(21,25;45;54) comprising means to regard the rest of the object as a buffer to be reproduced from iteration to 
iteration without reconstruction. 

10 

Patentanspruche 

1. Verfahren zum Bereitstellen einer auf einem Gitter (122; 47; 56) mit Gitterpunkten aufgebauten getreuen Rekon- 
15 struktion einer beobachteten Probe (1; 40; 51), wobei das Verfahren die Schritte aufweist: 

a) Bereitstellen mehrerer aufgezeichneter Beobachtungsdaten der Probe (10, 11; 40; 52) von jeweiis einem 
anderen Aspekt der Probe; 

b) Bereitstellen einer Varianz (S14) fur einzelne Beobachtungsgitterpunkte in jedem aufgezeichneten Beob- 
20 achtungsdatenelement; 

c) Bereitstellen einer Point Spread Function Oder Punktfaltungsfunktion (PSF) (18; 42; 53) in Abhangigkeit 
von Eigenschaften der Aufzeichnungsvorrichtung; 

d) Bereitstellen einer anfanglichen, frei gewahlten geschatzten Dichteverteilung der Probe (19; 44; 54) an den 
einzelnen Gitterpunkten; 

25 e) Bereitstellen einer unscharfen vorausbestimmten Dichteverteilung an den einzelnen Gitterpunkten unter 

Verwendung der anfanglichen geschatzten Verteilung, der Varianz und der Point Spread Function; 
f) Berechnen (12) durch ein iteratives Verfahren fur jede Iteration: 

* einer neuen geschatzten Verteilung der Probe an den Gitterpunkten unter Verwendung der vorausbe- 
30 stimmten Dichteverteilung und durch Vergleich zwischen der geschatzten Verteilung in der unmittelbar 

vorangehenden Iteration und den aufgezeichneten Beobachtungsdaten der Probe, und 

* einer neuen vorausbestimmten Dichteverteilung an den einzelnen Gitterpunkten basierend auf der neuen, 
geschatzten Verteilung unter Verwendung der Varianz und der Point Spread Function, die bezuglich der 
Dichteverteilung an den Gitterpunkten weniger unscharf ist als die vorausbestimmte Dichteverteilung in 

35 der unmittelbar vorangehenden Iteration; 

Fortsetzen der Iterationen, bis die Differenz zwischen der neuen geschatzten Verteilung und der unmittelbar 
vorangehenden geschatzten Verteilung kleiner ist als die vorgegebene Dichteverteilungsauflosungsbedin- 
gung an den Gitterpunkten, und Bereitstellen der letzten iterierten Dichteverteilung als die Rekonstruktion. 

40 

2. Verfahren nach Anspruch 1 , wobei die Unscharf e erzeugt wird, indem eine Fourier-Transformierte der geschatzten 
Struktur erzeugt wird und die Koeffizienten dieser Fourier-Transformierten mit den Fourier-Koeffizienten eines 
GauRschen und spharischen Filters multipliziert werden. 

45 3. Verfahren nach Anspruch 1 Oder 2, gekennzeichnet durch Normieren der unscharfen Struktur, bevor sie im nach- 
sten Iterationszyklus als die neue vorausbestimmte Verteilung verwendet wird. 

4. Verfahren nach einem der vorangehenden Anspruche, gekennzeichnet durch Ausschneiden einer Hullkurve von 
einem Objekt, die die zu rekonstruierende Probe darstelleh soil; Bereitstellen des Rests des Objekts als Puffer, 

50 der von Iteration zu Iteration ohne Rekonstruktion lediglich reproduziert werden soil. 

5. Verfahren nach einem der vorangehenden Anspruche mit den Schritten: 

• fur jede Iteration, urn eine schnellere Konvergenz zur endgultigen Rekonstruktion zu erhalten: 

55 

Berechnen einer reduzierten Chi-Quadrat-Statistik (C) als ein Maft dafiir, wie die gemessenen und be- 
rechneten Daten voneinander abweichen, unter Verwendung der unmittelbar vorangehenden, berechne- 
ten geschatzten Verteilung, der aufgezeichneten Beobachtungsdaten und der Varianz und des Gitters, 
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wahrend die Normierung der berechneten Dichteverteilung beibehalten wird; 

Berechnen einer Entropie (S) unter Verwendung der unmittelbar vorangehenden, berechneten geschatz- 
ten Dichteverteilung und der neuen vorausbestimmten Dichteverteilung; 

5 • Maximieren der Entropie zum Bereitstellen der glattesten erreichbaren Dichte unter der Einschrankung, daft 

die reduzierte Chi-Quadrat-Statistik gegen +1 gefuhrt wird, und Bereitstellen der reduzierten Chi-Quadrat- 
Statistik und der Entropie fur die Berechnung der im nachsten Iterationszyklus zu verwendenden neuen be- 
rechneten geschatzten Dichteverteilung. 

10 6. Verfahren nach einem der vorangehenden Anspruche, ferner mit den wahrend der Berechnung der neuen vor- 
ausbestimmten Dichteverteilung ausgefuhrten Schritten: 



Falten jeder neuen geschatzten Verteilung mit der Point Spread Function; 

Anwenden einer als Verschmierungsfunktion bezeichneten projizierten Point Spread Function auf die berech- 
15 nete Dichteverteilung; 

Anwenden einer Naherung derart, daft die Verschmierungsfunktion nur fur eine begrenzte Anzahl von Gitter- 
punkten der Probe grofter ist als null; und 

Erzeugen einer Struktur, deren Projektion, gefaltet mit der Point Spread Function, der Struktur der Beobach- 
tungsdaten angepaftt ist. 



20 



7. Verfahren nach Anspruch 6, ferner mit: 



Bereitstellen einer durch die Verschmierungsfunktion definierten Uberlappungsfunktion; 
Fuhren der reduzierten Chi-Quadrat-Statistik gegen +1 und Maximieren der Entropie unter Verwendung der 
25 Uberlappungsfunktion (S18). 

8. Verfahren nach Anspruch 7, wobei die mehreren aufgezeichneten Beobachtungsdaten der Probe durch die Auf- 
zeichnungseinrichtung (10, 11) bereitgestellt werden, die 2D-Projektionen der Probe in digitalisierter Form bereit- 
stellt und die darin enthaltenen Daten der Recheneinrichtung (12) zufuhrt, urn eine 3D-Bildrekonstruktion bereit- 

30 zustellen; 

gekennzeichnet durch 

Verarbeiten der Daten unter Verwendung einer Naherung, die darin besteht, daft die Verschmierungsfunktion 
nur fur eine begrenzte Anzahl von Gitterpunkten (122) in einer senkrecht zur Strahlrichtung angeordneten Schicht 
grofter ist als null, und durch Erzeugen der 3D-Struktur, deren Projektion, gefaltet mit der Point Spread Function, 
35 der beobachteten Struktur angepaftt ist (Gleichung 7). 

9. Verfahren nach einem der vorangehenden Anspruche fur eine 3D-Rekonstruktion einer Probe, dadurch gekenn- 
zeichnet, daft die Beobachtungen der Probe eine Serie verschieden geneigter 2-D-Beobachtungen der Probe 
sind. 

40 

10. Verfahren nach einem der Anspruche 1 bis 8 fur eine 3D-Rekonstruktion einer Probe durch Lichtaufzeichnungen, 
dadurch gekennzeichnet, daB die Beobachtungen von verschiedenen Aspekten der Probe eine Serie von 2-D- 
Beobachtungen sind, fokussiert auf verschiedene Tiefen der Probe. 

45 11. Verfahren nach einem der vorangehenden Anspruche, dadurch gekennzeichnet, daB die Rekonstruktion eine 
Rekonstruktion organischer Objekte ist, z.B. kleiner organischer Objekte, wie beispielsweise von Viruspartikeln 
Oder ahnlichen. 

12. Verfahren nach einem der Anspruche 1 bis 8, dadurch gekennzeichnet, daB die Rekonstruktion eine Musterre- 
50 konstruktion von auf einer Karte (50) bereitgestellten Probeelementen (51) mit geringfugig verschiedenen Struk- 

turen ist, urn eine sichere und eindeutige Identitat der Karte bereitzustellen. 

13. Verfahren nach einem der Anspruche 1 bis 6 fur eine 2D- Oder 1-D-Rekonstruktion, dadurch gekennzeichnet, 
daB die verschiedenen Aspekte der Probe Linien bzw. Punkte auf der Probe sind. 



55 



14. Vorrichtung zum Bereitstellen einer auf einem Gitter mit bestimmten Gitterpunkten aufgebauten getreuen Rekon- 
struktion einer beobachteten Probe eines Objekts, wobei die Vorrichtung aufweist: 
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eine Eingabeeinrichtung (10, 11; 40; 52) zum Bereitstellen von Beobachtungsdaten der Probe; 

eine Einrichtung zum Bereitstellen einer Varianz fur einzelne Beobachtungsgitterpunkte in jeder aufgezeich- 

neten Beobachtung; 

eine Einrichtung (18; 42; 53) zum Einfugen einer Point Spread Function (PSF) in Abhangigkeit von Eigen- 

5 schaften der Aufeeichnungsvorrichtung; 

eine Einrichtung (19; 44; 54) zum Bereitstellen einer geschatzten Dichteverteilung zum Bereitstellen einer 
anfanglichen, geschatzten, frei wahlbaren Dichteverteilung der Probe an den einzelnen Gitterpunkten; 
eine Einrichtung (21, 25; 45; 54) zum Bereitstellen einer vorausbestimmten Dichteverteilung zum Bereitstellen 
einer unscharfen vorausbestimmten Dichteverteilung an den einzelnen Gitterpunkten unter Verwendung von 

10 Daten der Einrichtung zum Bereitstellen einer geschatzten Dichteverteilung, der Varianz und der Point Spread 

Function; 

eine Recheneinrichtung (12) zum Berechnen durch ein iteratives Verfahren fur jede Iteration: 

* einer neuen geschatzten Dichteverteilung der Probe an den Gitterpunkten unter Verwendung der voraus- 
15 bestimmten Dichteverteilung und durch Vergleich zwischen der geschatzten Dichteverteilung in der un- 

mittelbar vorangehenden Iteration und den von der Eingabeeinrichtung erhaltenen Beobachtungsdaten 
der Probe, und 

* einer neuen vorausbestimmten Dichteverteilung an den einzelnen Gitterpunkten basierend auf der neuen, 
geschatzten Dichteverteilung unter Verwendung der Varianz und der Point Spread Function, die bezuglich 

20 der Dichteverteilung an den Gitterpunkten weniger unscharf ist als die vorausbestimmte Dichteverteilung 

in der unmittelbar vorangehenden Iteration; 

wobei die Recheneinrichtung (12) ferner aufweist: eine Einrichtung zum Fortsetzen der Iterationen, bis 
die Differenz zwischen der neuen geschatzten Dichteverteilung und der unmittelbar vorangehenden geschatz- 
25 ten Dichteverteilung kleiner ist als eine vorgegebene Bedingung; und eine mit der Recheneinrichtung verbun- 

dene Einrichtung (16; 46; 57) zum Darstellen des Rechenergebnisses als die Rekonstruktion. 

15. Vorrichtung nach Anspruch 14, wobei die Eingabeeinrichtung (10, 11) mehrere Aspekte einer Grofte (1) mifct; 

dadurch gekennzeichnet, daft 

30 

die Recheneinrichtung (12), urn eine schnellere Konvergenz zur endgultigen Rekonstruktion bereitzustellen, 
eine Einrichtung aufweist, die fur jeden Iterationszyklus folgende Berechnungen ausfuhrt: 
Berechnen einer reduzierten Chi-Quadrat-Statistik (C) als ein Mali dafur, wie die gemessenen und berechne- 
ten Daten voneinanderabweichen, unter Verwendung der unmittelbar vorangehenden, berechneten geschatz- 
35 ten Dichteverteilung, der aufgezeichneten Beobachtungsdaten und der Varianz und des Gitters, wahrend die 

Normierung der berechnete Dichteverteilung beibehalten wird; 

Berechnen einer Entropie (S) unter Verwendung der unmittelbar vorangehenden, berechneten geschatzten 
Dichteverteilung und der neuen vorausbestimmten Dichteverteilung; 

Maximieren der Entropie zum Bereitstellen der glattesten erreichbaren Dichte unter der Einschrankung, daft 
40 die reduzierte Chi-Quadrat-Statistik gegen +1 gefuhrt wird; und 

Bereitstellen der reduzierten Chi-Quadrat-Statistik und der Entropie fur die Berechnung der im nachsten Ite- 
rationszyklus zu verwendenden neuen berechneten geschatzten Dichteverteilung. 

16. Vorrichtung nach Anspruch 15, dadurch gekennzeichnet, daB die Recheneinrichtung (12) eine Einrichtung zum 
45 Berechnen einer als Verschmierungsfunktion bezeichneten projizierten Point Spread Function, die nur fur eine 

begrenzte Anzahl von Gitterpunkten der Probe grofier als null ist, und zum Erzeugen einer Struktur aufweist, deren 
Projektionen, gefaltet mit der Point Spread Function, der beobachteten Struktur angepaftt ist. 

17. Vorrichtung nach Anspruch 16, wobei die Eingabeeinrichtung aufweist: eine Vorrichtung (10; 52) zum Bereitstellen 
50 von Beobachtungsdaten von 2D-Projektionen der beobachteten Probe in digitaler Form und zum Zufuhreen der 

Daten zur Recheneinrichtung, urn eine getreue 3D-BiIdrekonstruktion bereitzustellen; 

dadurch gekennzeichnet, daB die Recheneinrichtung (12) ferner eine Einrichtung zum Verarbeiten der 
Daten unter Verwendung einer Naherung aufweist, die darin besteht, daB die Verschmierungsfunktion nur fur eine 
begrenzte Anzahl von Gitterpunkten in einer senkrecht zur Strahlrichtung angeordneten Schicht groBer ist als null, 
55 und zum Erzeugen der 3D-Struktur, deren Projektionen, gefaltet mit der Point Spread Function, der beobachteten 

Struktur angepaBt ist. 

18. Vorrichtung nach einem der Anspruche 14 bis 17, dadurch gekennzeichnet, daB die Eingabeeinrichtung ein 
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Mikroskop (10) ist, z.B. ein Elektronenmikroskop, ein Lichtmikroskop Oder ein konfokales Mikroskop. 

19. Vorrichtung nach einem der Anspruche 14 bis 17, dadurch gekennzeichnet, daB die Eingabeeinrichtung ein 
Kartenleser (52) zum Lesen eines Musters von auf einer Karte (50) bereitgestellten Probeelementen (51) mit ge- 

5 ringfugig verschiedenen Strukturen ist, urn eine Musterrekonstruktion zum Bereitstellen einer sicheren und ein- 

deutigen Identitat der Karte bereitzustellen. 

20. Vorrichtung nach einem der Anspruche 14 bis 17 fur eine 2D-Rekonstruktion, dadurch gekennzeichnet, daB die 
Eingabeeinrichtung eine Scanvorrichtung (40) zum Scannen eines 2D-Bildes ist, und daB die Projektionen Linien 

10 auf der Probe sind. 

21. Vorrichtung nach einem der Anspruche 14 bis 17, dadurch gekennzeichnet, daB die Eingabeeinrichtung (52) 
eine Einrichtung zum Aufzeichnen von Anderungen der GrofJe bezuglich einer vorgegebenen Variablen aufweist, 
und die Recheneinrichtung eine Einrichtung zum Rekonstruieren dergemessenen GroBe als Serie von MeBdaten 

15 aufweist. 

22. Vorrichtung nach einem der Anspruche 14 bis 21, dadurch gekennzeichnet, daB die Einrichtung (21 , 25; 45; 54) 
zum Bereitstellen der geschatzten und der vorausbestimmten Dichteverteilung eine Einrichtung aufweist, die eine 
Ausschnittshullkurve eines zu beobachtenden Objekts verarbeitet, wobei die Hullkurve die zu rekonstruierende 

20 Probe darstellt; und daB die Enrichtung (21 , 25; 45; 54) zum Bereitstellen der geschatzten und der vorausbestimm- 

ten Dichteverteilung eine Einrichtung aufweist, die den Rest des Objekts als Puffer betrachtet, der von Iteration 
zu Iteration ohne Rekonstruktion reproduziert werden soil. 

25 Reve nd icati o n s 

1. Procede pour fournir une reconstitution a haute fidelite construite sur une grille (122; 47; 56) ayant des points de 
grille d'un echantillon observe (1; 40; 51), ledit procede comprenant les etapes suivantes : 

30 a) fournir plusieurs donnees observees et enregistrees de I'echantillon (10, 11; 40; 52), chacune provenant 

d'un aspect different de I'echantillon; 

b) fournir une variance (S14) pour des points de grille d'observation individuels dans chaque donnee observee 
et enregistree; 

c) fournir une fonction d'etalement de point (PSF) (18; 42; 53) dependant des proprietes du dispositif d'enre- 
35 gistrement; 

d) fournir une distribution de densite estimee initiale, librement choisie, de I'echantillon (19; 44; 54) au niveau 
des points de grille individuels; 

e) fournir une distribution de densite de prejuge anterieur floue au niveau des points de grille individuels en 
utilisant la distribution estimee initiale, la variance et la fonction d'etalement de point; 

40 f) calcuter (12) dans un processus iteratif pour chaque iteration : 

* une nouvelle distribution estimee de I'echantillon au niveau des points de grille en utilisant la distribution 
de densite de prejuge anterieur et la comparaison entre la distribution estimee dans ('iteration precedente 
suivante et la donnee observee et enregistree de I'echantillon, 
45 * une nouvelle distribution de densite de prejuge anterieur au niveau des points de grille individuels sur la 

nouvelle distribution estimee moins floue concernant la distribution de densite au niveau des points de 
grille, en utilisant la variance et la fonction d'etalement de point, que la distribution de prejuge anterieur 
dans Iteration precedente suivante; 

50 continuer les iterations jusqu'a ce que la difference entre la nouvelle distribution estimee et la distribution 

estimee precedente suivante soit inferieure a une condition de distribution de resolution de densite pre- 
determines sur les points de grille, et 

fournir la derniere distribution de densite iteree en tant que reconstitution. 

55 2. Procede selon la revendication 1 , dans lequel le flou est obtenu en faisant une transformee de Fourier de la 
structure et en multipliant les coefficients de cette transformee par les coefficients de Fourier d'un filtre gaussien 
et spherique. 



25 



EP 0 885 430 B1 

3. Procede selon ia revendication 1 ou 2, caracterise par le fait de normaliser ladite structure floue avant qu'elle ne 
sott utiiisee comme nouvelle distribution de prejuge anterieur dans le cycle d'iteration suivant. 

4. Procede selon Tune quelconque des revendications precedentes, caracterise par le fait de decouper une enve- 
5 loppe d'un objet destine a etre I'echantillon a reconstituer, en foumissant le reste de I'objet en tant que tampon 

simplement destine a etre reproduit d'iteration en iteration sans reconstitution. 

5. Procede selon I'une quelconque des revendications precedentes, comprenant les etapes suivantes : 

10 • pour chaque iteration afin d'obtenir une convergence plus rapide vers la reconstitution finale : 

calculer une statistique en khi-deux reduite (C) comme mesure de la maniere dont les donnees mesurees 
et calculees different les unes des autresen utilisant la distribution estimee calculee immediatement pre- 
cedente, les observations enregistrees et ladite variance, et la grille, tout en maintenant la normalisation 
15 de la distribution de densite calculee, 

calculer une entropie (S) en utilisant la distribution de densite estimee calculee immediatement precedente 
et la nouvelle distribution de prejuge anterieur; 

• maximiser I'entropie pour obtenir la densite la plus lisse possible avec comme contrainte de faire tendre la 
20 statistique en khi-deux reduite vers +1, et fournir la statistique en khi-deux reduite et I'entropie pour le calcul 

de la nouvelle distribution de densite estimee calculee a utiliser dans le cycle d'iteration suivant. 

6. Procede selon I'une quelconque des revendications precedentes, comprenant en outre, pendant le calcul de la 
nouvelle distribution de densite de prejuge anterieur, les operations consistant a : 

25 

fournir a chaque nouvelle distribution estimee une convolution de la fonction d'etalement de point; 
introduce une fonction d'etalement de point projetee, appelee fonction de maculage, sur la distribution de 
densite calculee; 

faire ('approximation selon laquelle la fonction de maculage n'est superieure a zero que pour un nombre limite 
30 de points de grille dudit echantillon; et 

produire une structure dont la projection convoluee avec la fonction d'etalement de point va s'adapter a la 
structure des donnees observees. 
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7. Procede selon la revendication 6, comprenant en outre les operations consistant a : 
fournir une fonction de chevauchement definie par la fonction de maculage, 

faire tendre ladite statistique en khi-deux reduite vers +1, et maximiser I'entropie en utilisant ladite fonction de 
chevauchement (S18). 

40 8. Procede selon la revendication 7, dans lequel les differentes donnees d'observation enregistrees de I'echantillon 
sont donnees par le moyen d'enregistrement (10, 11) qui fournit des projections 2D sous forme numerisee de 
I'echantillon et transmet les donnees contenues en eiles au moyen de calcul (12) afin de donner une reconstitution 
d'image 3D, caracterise par le fait de 

traiter lesdites donnees en utilisant une approximation selon laquelle la fonction de maculage n'est superieure 

45 a zero que pour un nombre limite des points de grille (122) dans une couche perpend iculaire a la direction de 

faisceau, et en produisant la structure 3D dont la projection convoluee avec la fonction d'etalement de point va 
s'adapter a la structure observee (Equation 7). 

9. Procede selon Tune quelconque des revendications precedentes pour une reconstitution 3D d'un echantillon, ca- 
50 racterise en ce que lesdites observations de I'echantillon sont une serie d'observations 2D de I'echantillon incli- 

nees differemment. 

10. Procede selon I'une quelconque des revendications 1 a 8 pour une reconstitution 3D d'un echantillon au moyen 
d'enregistrements lumineux, caracterise en ce que lesdites observations de differents aspects de I'echantillon 

55 sont une serie d'observations 2D focalisees a differentes profondeurs de i'echantillon. 

11. Procede selon I'une quelconque des revendications precedentes, caracterise en ce que la reconstitution est 
effectuee pour des objets organiques, par exemple de petits objets organiques, tels que des particules virales ou 
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similaires. 

12. Procede selon Tune quelconque des revendications 1 a 8, caracterise en ce que la reconstitution est une recons- 
titution de motif d'elements echantillons (51) ayant des dessins legerement differents prevu sur une carte (50) afin 

5 de fournir une identite sure et unique de la carte. 

13. Procede selon Tune quelconque des revendications 1 a 6 pour une reconstitution 2D ou 1D, caracterise en ce 
que les differents aspects de I'echantillon sont des drottes et des points, respectivement, repartis sur I'echantillon. 

10 14. Appareil pour fournir une reconstitution a haute fidelite d'un echantillon observe d'un objet construite sur une grille 
ayant des points de grille determines, ledit appareil comprenant : 

un moyen d'entree (10, 11; 40; 52) fournissant des donnees observees de I'echantillon; 
un moyen pour donner une variance pour les points de grille d'observation individuels dans chaque observation 
15 enregistree; 

un moyen d'insertion (18; 42; 53) de fonction d'etalement de point (PSF) pour inserer une fonction d'etalement 
de point (PSF) dependant des proprietes du dispositif d'enregistrement; 

un moyen de distribution estimee (19; 44; 54) fournissant une distribution de densite estimee initiate, librement 
choisie, de I'echantillon sur les points de grille individuels; 
20 un moyen de distribution de prejuge (21 , 25; 45; 54) fournissant une distribution de densite de prejuge anterieur 

floue sur les points de grille individuels en utilisant le moyen de distribution estimee, la variance et la fonction 
d'etalement de point; 

un moyen de calcul (12), pour calculer dans un processus iteratif pour chaque iteration : 

25 * une nouvelle distribution de densite estimee de I'echantillon sur les points de grille en utilisant la distribution 

de densite de prejuge anterieur et la comparaison entre la distribution de densite estimee dans I'iteration 
precedente suivante et ladite donnee observee de I'echantillon provenant dudit moyen d'entree, 
* une nouvelle distribution de densite de prejuge anterieur sur les points de grille individuels sur la nouvelle 
estimee moins floue concernant la distribution de densite au niveau des points de grille, en utilisant la 

30 variance et la fonction d'etalement de point, que la distribution de prejuge anterieur dans I'iteration pre- 

cedente suivante; 

ledit moyen de calcul (12) comprenant en outre un moyen servant a continuer les iterations jusqu'a ce que la 
difference entre la nouvelle distribution de densite estimee et la distribution de densite estimee precedente 
35 suivante soit inferieure a une condition predeterminee; et un moyen (16; 46; 57) relie audit moyen de calcul 

pour presenter le resultat calcule en tant que reconstitution. 

15. Appareil selon la revendication 14, dans lequel ledit moyen d'entree (10, 11) mesure plusieurs aspects d'une 
quantite (1), caracterise par le fait que ledit moyen de calcul (12) pour chaque cycle d'iteration, afin d'obtenir une 

40 convergence plus rapide vers la reconstitution finale, comprend un moyen servant a calculer les valeurs suivantes : 

une statistique en khi-deux reduite (C) comme mesure de la maniere dont les donnees mesurees etcalculees 
different les unes des autres en utilisant la distribution estimee calculee immediatement precedente, les ob- 
servations enregistrees et ladite variance, et la grille, tout en maintenant la normalisation de la distribution de 
45 densite calculee, 

une entropie (S) en utilisant la distribution de densite estimee calculee immediatement precedente et la nou- 
velle distribution de prejuge anterieur; 

maximiser I'entropie pour obtenir la densite la plus lisse possible avec comme contrainte de faire tendre la statis- 
50 tique en khi-deux reduite vers +1, et fournir la statistique en khi-deux reduite et I'entropie pour le calcul de la 

nouvelle distribution de densite estimee calculee a utiliser dans le cycle d'iteration suivant. 

16. Appareil selon la revendication 15, caracterise en ce que ledit moyen de calcul (12) comprend un moyen pour 
calculer une fonction d'etalement de point projetee, appelee fonction de maculage, qui n'est superieure a zero que 

55 pour un nombre limite de points de grille dudit echantillon et produire une structure dont les projections convoluees 

avec la fonction d'etalement de point vont s'adapter a la structure observee. 

17. Appareil selon la revendication 16, dans lequel le moyen d'entree comprend un dispositif (10; 52) qui fournit des 
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donnees observees de projections 2D sous forme numerisee de rechantillon observe et transmet les donnees au 
moyen de calcul afin de donner une reconstitution d'image 3D a haute fidelite, 

caracterise en ce que ledit moyen de caicul (12) comprend en outre un moyen servant a traiter les donnees en 
utilisant une approximation selon laquelte ladite fonction de maculage n'est superieure a zero que pour un nombre 
5 limite de points de grille dans une couche perpendiculaire a la direction de faisceau, et en produisant la structure 

3D dont les projections convoluees avec la fonction d'etalement de point vont s'adapter a la structure observee. 

18. Appareil selon Tune quelconque des revendications 14 a 17, caracterise en ce que ledit moyen d'entree est un 
microscope (10), par exemple un microscope electronique, un microscope optique, ou un microscope a foyer 

10 commun. 

19. Appareil selon Tune quelconque des revendications 14 a 17, caracterise en ce que ledit moyen d'entree est un 
moyen formant lecteur de carte (52) pour lire un motif d'elements echantillons (51) ayant des dessins legerement 
differents prevu sur une carte (50) afin de fournir une identite sure et unique de la carte. 

15 

20. Appareil selon Tune quelconque des revendications 14 a 17 pour une reconstitution 2D, caracterise en ce que 
ledit moyen d'entree est un dispositif a balayage (40) servant a balayer une image 2D, et en ce que lesdites 
projections sont des droites reparties sur rechantillon. 

20 21. Appareil selon Tune quelconque des revendications 14 a 17, caracterise en ce que le moyen d'entree (52) com- 
prend un moyen servant a enregistrer une variation de la quantite par rapport a une variable predeterminee et en 
ce que ledit moyen de calcul comprend un moyen servant a reconstituer ladite quantite mesuree sous forme d'une 
serie de donnees de mesure. 

25 22. Appareil selon Tune quelconque des revendications 14 a 21, caracterise en ce que le moyen de distribution 
estimee et le moyen de distribution de prejuge (21, 25; 45; 54) comprennent un moyen servant a travailler sur une 
enveloppe decoupee dans un objet a observer, Penveloppe etant rechantillon a reconstituer; eten ce que le moyen 
de distribution estimee et le moyen de distribution de prejuge (21, 25; 45; 54) comprennent un moyen servant a 
considerer le reste de I'objet comme tampon a reproduce d'iteration en iteration sans reconstitution. 

30 
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